Analysis and Synthesis of Textured Motion: Particle, Wave and Cartoon Sketch

Natural scenes contain a wide range of textured motion patterns which are characterized by the movement of a large amount of particle and wave elements, such as falling snow, water waves, dancing grass, etc. In this paper, we present a generative method for modeling these motion phenomena and our method consists of four components: (1). A photometric model which represents an image as a linear superposition of image bases selected from a generic and over-complete dictionary. The dictionary contains Gabor and LoG bases for point/particle-elements and Fourier bases for wave-elements. These bases compete to explain the input images and transfer a raw image to a base (token) representation with O(10)-fold of dimension reduction. (2). A geometric model which groups adjacent bases and their motion trajectories into a number of basic moving elements – called “motons”. A moton is a deformable template in space-time representing a moving element, such as a snowflake. (3). A dynamic model which characterizes the motion of particles, waves, and their interactions, e.g., balls/leaves floating on water. The trajectories of these elements are coupled Markov chains. Given an input video sequence, a statistical learning algorithm computes a set of motons with their trajectories as hidden variables. It also learns the parameters that govern the geometric deformations and motion dynamics by maximum likelihood estimation (MLE). Consequently, novel sequences are synthesized easily from the learnt models. (4). A sketch model replaces the dictionary of Gabor/LoG and Fourier bases with symbolic sketches (token symbols). With the same generative model, the sketch model can render realistic and stylish cartoon animation. In this paper, cartoon sketch is viewed as a symbolic visualization and simplification of the inner representation in visual perception. The success of the cartoon animation, in turn, suggests that the generative model captures the essence of visual perception of textured motion.